New tricks from an old dog: An overview of TEI P5

This paper presents an update on the current state of development of the Text Encoding Initiative’s Guidelines for Electronic Text Encoding and Interchange. Since the last major edition in 2002, which saw the conversion of the Guidelines into XML, there has been substantial activity on adding new content in areas of particular interest to historical corpus builders. The TEI has also reinvented itself as a membership initiative and set up mechanisms for the continued development and maintenance of the Guidelines. We contrast "old" and "new" TEI, and give a brief overview of some recent technical enhancements to the system intended to facilitate expansion and customization of the scheme. 1 What did the TEI ever do for us? Monty Python fans will recall the scene in which a spokesperson for People’s Front of Judea (or was it the Judean Peoples’ Front?) asks rhetorically ‘But what did the Romans ever do for us?’, only to be overwhelmed with responses such as ‘Well, the roads... the sanitation... the cooking...’. In rather the same spirit, if asked what the TEI has contributed to the general area of digital scholarship, broadly defined as those who work in a critical and scholarly way with digitized textual materials, we can reasonably point to a not unimpressive list of modest achievements. Each of the four major published drafts of the TEI’s Guidelines for Electronic Text Encoding and Interchange (TEI 1990, 1992, 1994, 2000) has consolidated its position of pre-eminence within an increasingly active field. The TEI Guidelines describe and define nearly 500 well-documented textual distinctions, covering the encoding needs of many academic fields in exhaustive detail. The Guidelines are further complemented by a detailed technical implementation, which is based entirely on the use of XML standards and can thus be supported by any of a wide and continually expanding range of tools. The whole is organized within a modular and extensible architecture, with an independent academic governance, and a remarkably wideranging and varied community of practice. Approaching its second decade, an eternity in the world of information technology, the TEI might be expected to feel moderately complacent at its success, but also to be preparing to retire gracefully from the scene. Such a view might also be encouraged by the observation that the TEI had its origins in a specific academic research project, which although well funded initially, had been unable (like many other very successful academic research projects) to agree on either an exit strategy or a viable business model for its continuation. Although the TEI had succeeded beyond expectation in codifying — even perhaps in determining— common encoding practices, it lacked any formal way of coping with their ongoing rapid evolution. Its licencing and development practices were somewhat uncertain. Despite its widespread take-up, it was perceived in some quarters as being unmanageably complex except by an inner circle of devotees, and simultaneously (even sometimes by the same people) as being too simple for real scholarly work. Where were the tools to do new and interesting things with TEI-encoded texts and how might their production best be fostered? Such criticisms reflect above all the lack of an active maintenance and development community for the TEI. A roadmap addressing these concerns was announced in the year 2000 with the launch of a new governance structure for the TEI, re-organised as a membership consortium. Aside from constitutional issues, this announcement also mapped out a clear path for the future technical development of the Guidelines. The first step would be to convert the whole system to use XML rather than SGML as its means of expression; this was accomplished with the current P4 release in 2002. Backwardscompatibility was an explicit design goal for that release, and therefore no attempt was made to remove or revise much which might be considered obsolescent, nor to address any of the many new areas of digital activity where the TEI approach might be of use or relevance. As the preface to P4 indicates, ‘these tasks require the existence of an informed and active TEI Council to direct and validate such 1 Dagstuhl Seminar Proceedings 06491 Digital Historical CorporaArchitecture, Annotation, and Retrieval http://drops.dagstuhl.de/opus/volltexte/2007/1042